SOBA: Secrecy-preserving Observable Ballot-level Audit 
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Abstract 

SOBA is an approach to election verification that pro- 
vides observers with justifiably high confidence that the 
reported results of an election are consistent with an audit 
trail ("ballots"), which can be paper or electronic. SOBA 
combines three ideas: (1) publishing cast vote records 
(CVRs) separately for each contest, so that anyone can 
verify that each reported contest outcome is correct, if 
the CVRs reflect voters' intentions with sufficient accu- 
racy; (2) shrouding a mapping between ballots and the 
CVRs for those ballots to prevent the loss of privacy that 
could occur otherwise; (3) assessing the accuracy with 
which the CVRs reflect voters' intentions for a collection 
of contests while simultaneously assessing the integrity 
of the shrouded mapping between ballots and CVRs by 
comparing randomly selected ballots to the CVRs that 
purport to represent them. Step (1) is related to work 
by the Humboldt County Election Transparency Project, 
but publishing CVRs separately for individual contests 
rather than images of entire ballots preserves privacy. 
Step (2) requires a cryptographic commitment from elec- 
tions officials. Observers participate in step (3), which 
relies on the "super-simple simultaneous single-ballot 
risk-limiting audit." Step (3) is designed to reveal rel- 
atively few ballots if the shrouded mapping is proper and 
the CVRs accurately reflect voter intent. But if the re- 
ported outcomes of the contests differ from the outcomes 
that a full hand count would show, step (3) is guaranteed 
to have a large chance of requiring all the ballots to be 
counted by hand, thereby limiting the risk that an incor- 
rect outcome will become official and final. 

1 Introduction and background 

The majority of Americans now vote electronically, ei- 
ther on machine-counted paper ballots or on Direct 
Recording Electronic (DRE) machines. Electronic vot- 
ing offers advantages over hand counts and lever ma- 
chines, but it poses challenges for determining whether 



votes were recorded and counted correctly. A wide range 
of security vulnerabilities and other flaws have been doc- 
umented in contemporary voting equipment. The 2007 
"Top-to-Bottom Review" of the systems used in Califor- 
nia found that all the systems had "serious design flaws" 
and "specific vulnerabilities, which attackers could ex- 
ploit to affect election outcomes' ' iBowenl(2()07ll . While 
some of these vulnerabilities can be mitigated, the under- 
lying verification challenge is formidable. As Rivest and 
Wack comment, "complexity is the enemy of security," 
and demonstrating that any complex system is free of 
faults may be impossible or infeasible IIRivest and Wack , 
l2006ll . 

Electronic voting systems have failed in real elec- 
tions. In the 2004 general election in Carteret County, 
North Carolina, over 4,000 votes were lost irretriev- 
ably due to a programming error that affected UniLect 
Patriot voting ma chines, casting doubt on a statewide 
election outcome OBonnen 1200411 . More controversially, 
in the 2006 general election, ES&S iVotronic DREs 
in Sarasota County, Florida did not record a vote for 
U.S. House for about 15% of voters — far more than can 
plausibly be attributed to intentional undervoting. Inad- 
verte nt undervotes were prob a bly decisive in that con - 
test llAsh and LampertiL l2008t iMebane and DillL l2007ll . 
Hypotheses explaining these undervotes include voter 



confusion caused by poor ba llot layout IFrisina et al. 



2008ll and machine failure iGarbeii l2008l iMebane 



200911 . Unfortunately, the forensic evidence generated 



by the voting systems was inadequate to determine the 
cause of the undervotes or the intentions of the voters. 

Voter-marked paper ballots provide a clearer record 
of what voters did and more evidence about voter in- 
tent, but by themselves do not solve the election verifica- 
tion problem. In 2005, Hani Hursti repeatedly demon- 
strated the ability to "hack" o ptical scan co unts when 
given access to a memory card 1 Zetteri 2005 1 . In a June 
2006 primary election in Pottawattamie County, Iowa, 
incorrectly configured optical scanners miscounted ab- 
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sentee ballots in every contest, altering two outcomes. 
The county auditor ordered a hand recount, which cor- 
rected the errors [Flaherty, 2006]. Similar errors in other 
elections may have altered outcomes without ever being 
detected. Even when scanners work correctly, their re- 
sults may differ materially from voter intent. Consider 
the 2006 U.S. Senate contest in Minnesota, where Al 
Franken beat Norm Coleman in a hand recount largely 
because of ballots where the human interpretation dif- 
fered from the machine interpretation^ 

1.1 Software independence 

Computerized election equipme nt cannot be i nfallible, 
so Riyest and Wack [2006] and iRivestI EOOSll suggest 
that voting systems should be software-independent. A 
voting system is software-independent "if an undetected 
change or error in its software cannot cause an unde- 
tectable change or error in an [apparent] election out- 
come." This idea can be generalized to define indepen- 
dence from hardware and from elections personnel, lead- 
ing to so-called end-to-end verifiable election technolo- 
gies. However, end-to-end technology may require fun- 
damental changes in current voting processes. 

The outcome of a contest is the set of winners, not the 
exact vote counts. The apparent outcome of a contest 
is the winner or winners according to the voting system. 
The correct outcome of a contest is the winner or win- 
ners that a full hand count of the "audit trail" would find. 
The audit trail is assumed to be an indelible record of 
how voters cast their votes. It might consist of a com- 
bination of voter-marked paper ballots, voter receipts, a 
voter-verifiable paper audit trail (VVPAT), and suitable 
electronic records. 

This definition of "correct" is generally a matter of 
law. It does not necessarily imply that the audit trail 
is inviolate (nor that the outcome according to the audit 
trail is the same as the outcome according to how voters 
originally cast their ballots); that there is no controversy 
about which records in the audit trail reflect valid votes; 
that human observers agree on the interpretation of the 
audit trail; that the actual hand counting is accurate; nor 
that repeating the hand count would give the same an- 
swer If there is no audit trail, defining what it means for 
the apparent outcome to be correct requires hypothetical 
counterfactuals — ^but for the fault in the voting system, 
what would the outcome have been? 

Software independence means that errors that cause 
apparent outcomes to be wrong leave traces in the audit 
trail. But software independence does not guarantee any 
of the following: 



1. that no such traces will occur if the apparent out- 
come is correcH 

2. that those traces will be noticed or acted upon 

3. that the cost of looking through the audit trail for 
those traces is affordable 

4. that, in principle, there is a way to correct the appar- 
ent outcome without holding another election 

5. that, in practice, the audit trail was preserved and 
protected well enough to determine the outcome ac- 
cording to how the voters originally cast their bal- 
lots 

The penultimate pr operty is guaranteed by stron g soft- 
ware i ndependence. iRivest and Wackl ll2006ll and lRivest 
i2008li define a voting system to be strongly software- 
independent if an undetected change or error in its soft- 
ware cannot cause an undetectable change or error in an 
[apparent] election outcome, and moreover, a detected 
change or error in an [apparent] election outcome (due to 
change or error in the software) can be corrected without 
re-running the election. Having an audit trail does not 
guarantee that anyone will dig through it to see whether 
there is a problem or to correct the outcome if the out- 
come is wrong. Strong software independence does not 
correct anything, but it is an essential ingredient for a 
system to be self-correcting. 

Compliance audits can be used to assess whether the 
last property listed above holds: Given that the election 
used a strongly software-independent voting system, did 
it adhere to procedures that should keep the audit trail 
sufficiently accurate to reconstruct the outcome accord- 
ing to how voters cast their ballots? Strong evidence that 
such procedures were followed is strong evidence that 
the legally correct outcome — what a full hand count of 
the audit trail would show — is the same as the outcome 
according to how the voters originally cast their ballots. 
As we discuss below in section HI we believe that com- 
pliance audits should always be required: If the election 
fails the compliance auditj^ there is no assurance that 
even a full hand count of the audit trail would show the 
outcome according to how the voters really voted. Be- 
low, we assume that the election has passed a compliance 
audit. 



' The 2000 presidential election may have been decided by differ- 
ences between the machine interpretation of certain Florida optical scan 
ballots and the likely human interpretation lKeatind . l2002ll . 



^False alarms are possible. An analogy is that if a tamper-evident 
seal shows that a package has been opened, it does not follow that the 
package contents have been altered. 

^"Failure" means failure to find strong evidence that such proce- 
dures were followed, rather than finding evidence that such procedures 
were not followed. 
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1.2 Vote tabulation audits 

Vote tabulation audits compare reported vote subtotals 
for subsets of ballots ("audit units") with hand counts of 
the votes for each of those subsets. Audit units have to be 
subsets for which the voting system reports vote subto- 
tals. Most present U.S. audits use audit units that consist 
of all the ballots cast in individual precincts or all the bal- 
lots tabulated on individual voting machines. Generally, 
audit laws do not have provisions tha t would lea d to cor- 
recting incorrect electoral outcomes llHall et al. Eb09l FI 
A risk-limiting post-election audit uses the au- 
dit trail to guarantee that there is a large, pre- 
specified probability that the audit will correct the 
apparent outcome if the apparent outcome is wrong. 
Risk - limiting audits are w idely considered best prac- 



tice ILindeman et al.L 1200811 . Risk-limiting audits have 



been endorsed by the American Statistica l Associa- 
tion f American Statistical Association, 2010ll . the Bren- 
nan Center for Justice, Common Cause, the League of 
Women Voters, and Verified Voting, among others. Cal- 
ifornia AB 2023 (2010), r e quires a pilot of risk-limiting 
audits in 2011 OSaldafiaL l2010ll . Colorado Revised 
Statutes §1-7-515 calls for implementing risk-limiting 
audits by 2014. 

The first method for condu ct ing ri sk-limiting 



audits was proposed by I Stark I 2008all: 
ous improvem ents have been m ade 
|2009 b d c; Mir atrix and Starki 12009: Stark, ,2010 b |. See 
also ICheckow av et al.l 2010 1 . Risk-limiting audits limit 



.. numer- 
[ Stark, '2008b', 



the risk of failing to correct an outcome that is wrong. 
The risk limit is 100% minus the minimum chance that 
the audit corrects the outcome. If the outcome is correct 
in the first place, a risk-limiting audit cannot make it 
wrong; but if the outcome is wrong, a risk-limiting 
audit has a large chance of correcting it. Hence, the 
probability that the outcome according to a risk-limiting 
audit is the correct outcome is at least 100% minus the 
risk Umit. 

For systems that are strongly software-independent, 
adding a risk-limiting audit addresses the second condi- 
tion above: It ensures a large, pre-specified probability 
that the traces will be noticed and will be used to correct 
the apparent outcome if the apparent outcome is wrong. 

1.3 Our goal 

Our goal in this work is to sketch a personally verifiable 
privacy-preserving P-resilient canvass framework. We 
must first say what this means. 



A canvass framework consists of the vote-tabulation 
system together with other human, hardware, software, 
and procedural components of the canvass, including 
compliance and vote-tabulation audits. A canvass frame- 
work is resilient with probability P or P-resilient if the 
probability that the outcome it givefl is the correct out- 
come is at least P, even if its software has an error, short- 
coming, or undetected change@ Resilience means that 
the framework tends to recover from faults. If a can- 
vass framework is f -resilient, either the outcome it gives 
when all is said and done is correct, or something oc- 
curred that had probability less than I — P. The canvass 
framework that results from performing a risk-limiting 
audit on a strongly software-independent voting system 
that passes a compliance audit is P-resilient, with P equal 
to 100% minus the risk limit. If the system fails the com- 
pliance audit, the framework should not declare any out- 
come. Instead, the election should be re-run. 

Even if a canvass framework is P-resilient, in prac- 
tice the public might not trust the system unless they 
can observe crucial steps, especially the audit. The mere 
right or opportunity to observe the audit will not engen- 
der much trust if — as a practical matter — no single per- 
son or small group could observe all the steps that are 
essential to ensuring the accuracy of the final result. For 
instance, if a vote-tabulation audit takes ten teams of au- 
ditors working in separate offices four days to complete, 
it would take a large team of independent observers — 
with lots of free time and long attention spans — to verify 
that the audit was carried out correctly. The longer an 
audit takes and the more people required to carry out the 
audit, the more opportunities there are to damage the au- 
dit trail, and the harder it is for an observer to be satisfied 
that the audit has been conducted correctly. 

We define a canvass framework to be personally verifi- 
able P-resilient if it is P-resilient and a single individual 
could, as a practical matter, observe enough of the pro- 
cess to have convincing evidence that the canvass frame- 
work is in fact P-resilient. 

The transparency required for a canvass framework to 
be personally verifiable can impact privacy. For instance, 
publishing images of all the ballots cast in an electiorQ 
might give the individuals compelling evidence that the 
vote tabulation system found the correct outcome, since 
the images allow people to count the votes themselves — 



^For instance, under New York law. each county determines inde- 
pendently whether its audit in a particular contest must be expanded. 
This provision means that a connect outcome might be changed to an 
incorrect outcome even if the conduct of the audit is formally flawless. 



^As discussed in section |4] to be P-resilient, a canvass framework 
should refrain from giving any outcome at all if some preconditions are 
not met. 

*The probability comes from the overall voting system, in our case 
from the fact that the audit relies on a random sample. The probabil- 
ity does not come from treating votes, voters, or election outcomes as 
random, for instance. 

^ There also needs to be proof that the images are sufficiently com- 
plete and accurate to determine the correct outcome. 



3 



at least to the extent that voter intent is unambiguous^ 
But publishing ballot images can facilitate vote-selling 
and coercion and can compromise privacy, because vot- 
ers can deliberately or accidentally reveal their identities 
through marks on the ballots including idios yncrasies of 
how individuals fill in bubbles [Calandrino e t al.l 1201 1 1 



or even the fi ber structure of the pape r on which the bal- 
lot is printed fCalan drinoetalll2009ll PI 

A lesser but substantial degree of transparency is 
conferred by publishing cast vote records (CVRslFl 
enabling anyone to verify that the contest outcomes 
are correct — if the CVRs are accur ate. However , 
as IPopoveniuc and Stanton 1 2007 1 and Rescorlal ll2009ll 
point out, publishing CVRs also can aid vote-selling or 
coercion because of the potential for pattern voting. One 
typical sample ballot (from Tulsa, Oklahoma) contains 
18 contests with over 589,000 possible combinations if 
a voter votes in every contest, or over 688 million com- 
binations allowing for undervotes. Thus, a voter could 
be instructed to vote for the preferred candidate in one 
contest, and to cast a series of other votes that would al- 
most certainly (especially within a precinct), confirm the 
voter's identity if all of the voter's selections were pub- 
lished. Hence, publishing whole-ballot CVRs for large 
numbers of ballots improves transparency but can sacri- 
fice privacy. 

When there is not strong evidence that the apparent 
outcome is correct, risk-limiting audits can require ex- 
amining the entire audit trail, potentially exposing all the 
ballots to public scrutinyl3 If the apparent outcome is 
wrong, such exposure is necessary in order to correct 
the outcome. Therefore, if a risk-limiting audit is to 
be personally verifiable, there may be occasions where 
compromising privacy is unavoidable. But minimizing 
the number of ballots or whole-ballot CVRs that are 
routinely exposed helps protect privacy, impeding vote- 



"Verification methods like Humboldt County Election Transparency 
Project (see below) involve publishing digital images of all the ballots. 

'There are arguments that images of ballots should be published 
anyway — that transparency is more important than privacy. In juris- 
dictions that perniit voting by mail, there is an opportunity to confimi 
how someone votes for the purpose of vote-selling or coercion; indeed, 
someone could fill out another's ballot. Whether publishing images of 
ballots would change the rate of vote-selling or coercion substantially 
is the subject of some debate. 

'"in the 2002 FEC Vot ing System Stan- 
dards iFederal Election CommissionL l2002ll . these were 
called "ballot images"; however, the term CVR has been 
used in more recent EAC Volunt a ry V oting System Guide- 
Unes lElection Assistance CommissionL l2005ll ; we prefer the latter 
term because it does not suggest an actual image but rather a record of 
the interpretation of the system's interpretation of the ballot. And what 
matters is the system's interpretation of the ballot as a set of votes. 

' ' One could have a risk-limiting audit that, if it had not tenninated 
after some fraction of the ballots had been examined, triggered a hand 
count of the remaining ballots, but did not allow the public to observe 
that hand count. But then why should the public trust that the hand 
count was accurate? 



selling and coercion. 

We define a canvass framework to be personally ver- 
ifiable privacy-preserving P-resilient if it is personally 
verifiable P-resilient and it does not sacrifice privacy un- 
necessarily. Neither personally verifiable nor privacy- 
preserving is a mathematically precise characteristic, 
while P-resilience is. 

The contribution of the present work is to sketch a 
personally verifiable privacy-preserving P-resilient vot- 
ing system. We assume, as a foundation for building 
this system, that we are starting with a strongly software- 
independent voting system with an audit trail that corre- 
sponds to individual ballots. Moreover, we assume that a 
compliance audit has determined that the audit trail gen- 
erated by the system is sufficiently trustworthy to reflect 
the correct outcomes of the contests. We augment the 
system with procedures and data structures that make it 
possible for an individual observer to gain compelling 
evidence that either the outcomes are correct, or some- 
thing very unlikely occurred — that is, that the overall 
canvass framework is P-resilient. Unless some of the 
apparent outcomes are wrong or a margin is extremely 
small, gathering that evidence will generally involve ex- 
posing only a tiny percentage of ballots and whole-ballot 
CVRs. 

In essence, our method adds a special risk-limiting 
audit to a strongly software-independent voting system 
(one that has had a compliance audit to ensure that its 
audit trail is intact). Since one person cannot be in two 
places at the same time, the procedure cannot be person- 
ally verifiable if it involves auditing a multi -jurisdictional 
contest in different jurisdictions simultaneously; it would 
then be necessary to trust confederates to observe what is 
happening elsewhere. The next few sections outline ele- 
ments of this risk-limiting audit. 

2 Ballot-level risk-limiting audits 

One key to keeping the process personally verifiable (by 
keeping amount of observation required low) and to pro- 
tecting privacy (by exposing as few ballots as possible 
to observers) is to audit the record at the level of indi- 
vidual ballots, rather than large batches of ballots such 
as precincts. The fewer ballots there are in each audit 
unit, the smaller the expected counting burden for risk- 
limiting audits tends t o be— when the elect oral outcome 
is con-ect (see, e.g., llStarkl l2009al bOlO a'"?!). A vote- 



tabulation audit based on checking the CVRs of individ- 
ual ballots against a human interpretation of those ballots 
is often called a "ballot-level audit," a "single-ballot au- 
dit," or a "ballot-based audit." Because they reduce the 
time it takes to audit and the number of ballots involved, 
ballot-level risk-limiting audits are especially amenable 
to personal verification. 



4 



Ballot-level audits are extremely efficient statistically, 
but they are not simple to implement using current vot- 
ing systems. To perform a ballot-level audit, there must 
be a way to identify each ballot uniquely, for instance, 
a serial number on a paper ballot, or identifying the bal- 
lot by its location: "the 17th ballot in deck 152 scanned 
by scanner C," for instanceQ There must also be a way 
to match each ballot to its CVR. Some commercial vot- 
ing systems do not generate or do not store CVRs for 
individual ballots. Other voting systems record individ- 
ual CVRs, but are designed make it difficult or impos- 
sible to match individual CVRs to the ballots they pur- 
port to represent. In some cases, audit trails have identi- 
fiers that can be used to find the corresponding CVRs; 
this method was use d for part of a 200 8 audit in Ea- 
gle County, Colorado | Branscornbl l2008ir^ and a ballot- 
level risk-limiting audit in Orange County, California, in 
2011 [RB. Stark, personal communication, 2011]. How- 
ever, to protect privacy, most paper ballots do not have 
identification numbers. In a 2009 pilot ballot-level audit 
in Yolo County, California, Stark 1 2009c] exploited the 
fact that the CVRs and the physical ballots were in the 
same order The scanned images associated with each 
CVR in the audit sample were compared with the physi- 
cal ballots to check the accuracy of the CVRs. 



Calandrino et al.l 11200711 describe an approach to elec- 



tion verification that involves imprinting ballots with 
identification numbers and scanning the ballots with a 
"parallel" system in addition to the system of record. 
The parallel system derives its own CVRs, from which 
the apparent contest outcome can be determined inde- 
pendently. The accuracy of the unofficial CVRs and of 
the imprinting process is then assessed by a ballot-level 
audit. 

Since 2008, the Humboldt County Election Trans- 
parency Project (Humboldt County ETP) has experi- 
mented with publishing ballot images and independently 
tabulating CVRs extracted from those images. Using 
commercially available equipment, Humboldt County 
ETP rescans paper ballots after embossing them with 
serial numbers. Then, open-source software is used to 
form CVRs from the digital images. Humboldt County 
ETP has processed ballots for six elections and published 
scanned ballot images as well as its version of the CVRs 
for some of them. The results based on their re-scans 



'^If an identifier is printed on paper ballots, the printing should occur 
after the voter casts his or her vote and the ballots are co-mingled. If 
the identifier is printed before the voter casts his or her vote, privacy 
could be compromised. 

" Optical-scan ballots as well as DRE paper audit trails can have 
identifiers. For instance, in Boulder County, Colorado, the Hart Ballot 
Now system is configured to print unique identifiers and bar codes on 
each ballot. In Orange County, Califomia, ballots for the Hart Ballot 
Now system have non-unique identifiers and bar codes (numbered 1- 
2500, then repeating). 



generally have agreed well with the original results, with 
one important exception: The Humboldt County ETP 
analysis of the November 2008 election uncovered a de- 
fect in the election management software that led the re- 
sults of an entire ballot batch to be silently discarded! 

The Clear Ballot Group, inspired in part by Humboldt 
County ETP, is developing a system that, in its words, 
could permit election outcomes to be "thoroughly and 
transparently verified within 36^8 hours after the polls 
close." Neither the Humboldt County ETP nor Clear 
Ballot Group currently incorporate risk-limiting auditsF^ 
but the parallel scans their systems perform facilitate 
ballot-level r isk-limiting audit s , alon g the general lines 
proposed bv ICalandrino et al.l ll2007ll . If the system of 
record and the parallel system agree on the set of win- 
ners, a risk-limiting audit of the parallel system transi- 
tively confirms the outcome according to the system of 
recordO 



3 A privacy-preserving audit 

The method we propose here presupposes that CVRs are 
available, either from the system of record or from a par- 
allel system. It publishes all the data contained in the 
CVRs in a form that (1) still permits all observers to 
check the contest outcomes on the assumption that the 
CVRs are accurate, (2) does not compromise privacy, and 
(3) enables the CVRs to be checked against the audit trail 
while minimizing the loss of privacy. 

In SOBA, election officials make a cryptographic 
commitments to the full set of CVRs by publishing the 
CVRs separately for each contest, disaggregating the bal- 
lots (we call these contest-CVRs or CCVRs in contrast to 
whole-ballot CVRs), and a shrouded link between each 
CCVR and the ballot it purports to represent. Splitting 
the CVRs into CCVRs and obfuscating the identity of the 
ballot from which each CCVR comes eliminates some of 
the information required to identify a voter's ballot style 
or to use pattern voting to signal the voter's identityFl 
This makes the procedure privacy-preserving. But it re- 
tains enough information for any observer to check that 



'''clear Ballot Group is adding support for risk-limiting audits to 
their software [L. Moore, personal communication, 2011]. 

'^This is true as long as the systems agree on the set of winners, 
even if they disagree about vote totals or margins. For instance, suppose 
candidate A defeats candidate B by one percentage point in the original 
returns, and by ten points according to the parallel system. Such a large 
discrepancy might justify close scrutiny, but a risk-limiting audit of the 
results of the parallel system would still provide strong evidence that A 
defeated B, or would lead to a full hand count to set the record straight. 

'^See http : //en. wikipedia. org/wiki/Comiiiitment_schemej 
Cryptographic commitments have two important properties, the bind- 
ing property and the hiding property, discussed in section 13^21 

"of course, if there is a contest in which few voters are eligible to 
vote, eligibility itself is a signal. 
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the apparent outcome agrees with the outcome accord- 
ing to the CCVRs, for each contest. That is, there is a 
known algorithm (the winner algorithm^) that observers 
can apply to the published CCVRs to calculate the cor- 
rect outcome of every contest — ^provided the CCVRs re- 
flect the ballots (more generally, audit trail) accurately 
enough. This is part of making the procedure personally 
verifiable. Loosely speaking, the required level of accu- 
racy depends on the number of CVRs that must have er- 
rors for the apparent outcome to be wrong The fewer 
ballots that need to be changed to affect the outcome, 
the larger the sample generally will need to be to attain 
a given level of confidence that the apparent outcome is 
correct. 

The CCVRs might fail to be sufficiently accurate be- 
cause 

• At least one CCVR and the ballot it purports to rep- 
resent do not match because human and machine in- 
terpretations of voter intent differ (for instance, be- 
cause the voter marked the ballot improperly). This 
is a failure of the generation of CCVRs. 

• At least one CCVR does not in fact correspond to 
any ballot. It is an "orphan." This is a failure of the 
mapping between ballots and CCVRs. 

• More than one CCVR for the same contest is 
mapped to the same ballot. It is a "multiple." This 
is also a failure of the mapping between ballots and 
CCVRs. 

• There is no CCVR corresponding to some voting 
opportunity on a ballot. 

A failure of the mapping might be the more distressing 
source of error, since it is a failure on the part of the 
election official, but we must ensure (statistically) that — 
together — all sources of error did not combine to cause 
the outcome to be wrong. SOBA uses a risk-limiting au- 
dit to assess statistically whether the winners according 
to the full audit trail differs from the winners according 
to the CCVRs, for all contests under audit, taking into ac- 
count all sources of error. If the outcome according to the 
CCVRs is incorrect, the audit is very likely to proceed to 
a full hand count of the audit trail, thereby revealing the 
correct outcome. This provides P-resilience. 

To make the risk-limiting audit possible, elections of- 
ficials are required to publish another file, the ballot style 

'^For first-past-the-post contests, the winner algorithm just finds 
who has the most votes. Other voting schemes, such as instant-mnolf 
voting (IRV) or ranked choice voting (RCV), have more complicated 
winner algorithms. 

"in plurality voting, this is the margin or the set of margins between 
each (winner, loser) pair Defining the margins for IRV a nd calculatin g 
them for a given set o f reported results is not simple. See ICarvl |20I ill ; 
lMagrinoet5]|20Illl . 



file, which contains ballot identifiers and lists the con- 
tests each of those ballots contains. It does not contain 
the voters' selections. 

The risk-limiting technique we propose is the 
super-simple simultaneous single-ballot risk-limiting au- 
dit [Stark, .2010bl . It is not the most efficient ballot- 
level audit, but the calculations it requires can be done 
by hand, increasing transparency. It involves drawing 
ballots at random with equal probability; some more ef- 
ficient audits require using different probabilities for dif- 
ferent ballots, which is harder to implement and to ex- 
plain to the public. Moreover, this technique allows a 
collection of contests to be audited simultaneously using 
the same sample of ballots. That can reduce the number 
of randomly selected ballots that must be located, inter- 
preted, and compared with CVRs, decreasing the cost 
and time required for the audit and thereby increasing 
transparency. 

The following subsections give more technical detail. 

3.1 Data framework and assumptions 

We assume that the audit trail consists of one record per 
ballot cast. There are C contests we wish to assess. The 
contests might be simple measures, measures requiring 
a super-majority, multi-candidate contests, or contests of 
the form "vote for up to W candidates.'^ We refer to 
records in the audit trail as "ballots." A ballot may be an 
actual voter-marked paper ballot, a voter- verifiable paper 
audit trail (VVPAT), or a suitable electronic record. 

There are ballots in the audit trail that each con- 
tain one or more of the C contests. Each ballot can be 
thought of as a list of pairs, one pair for each contest on 
that ballot. Each pair identifies a contest and the voter's 
selection(s) in that contest, which might be an undervote 
or a vote for one or more candidates or positions. Ex- 
amining a ballot by hand reveals all the voter's selections 
on that ballot; we assume that there is no ambiguity in 
interpreting each voter's intentions from the audit trail. 

Before the audit starts, the voting system must report 
results for each of the C contests. The report for contest 
c gives Nc, the total number of ballots cast in contest c 
(including undervotes and spoiled ballots), as well as the 
number of valid votes for each position or candidate in 

contest c. Let M ^Nx+Ni^ hA^c denote the total 

number of voting opportunities on the ballots. We as- 
sume that the compliance audit assures us (e.g., through 
ballot accounting) that the reported values of Nc are accu- 
rate, and that the audit trail is trustworthy. In the present 
work, we do not consider attacks on the audit trail. 



We do not specifically consider instant-runoff voting or ranked- 
choice voting here. Risk-limiting methods can be extended to such 
voting methods, but the details are complex. 
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There is a pubHshed "ballot style file." Each line in the 
ballot style file lists a ballot identifier and a list of con- 
tests that ballot is supposed to contain. The ballot iden- 
tifier uniquely identifies a ballot in the audit trail. The 
identifier could be a number that is printed on a paper 
ballot or unambiguous instructions for locating the bal- 
lot (e.g., the 275th ballot in the 39th deck). There should 
be lines in the file, and the ballot identifiers should 
be unique. Because the ballot style file is published, indi- 
vidual can check this for themselves. Moreover, individ- 
uals can check whether the number of lines in the ballot 
style file that list contest c equals A^^ , the total number of 
ballots the system reports were cast in contest c. 

Before the audit starts, the voting system or a paral- 
lel system has produced a CVR for each ballot. These 
are not published as whole-ballot CVRs. Rather, the 
CVRs are split by contest to make contest-specific CVRs 
(CCVRs) that contain voters' selections in only one con- 
test. Each whole-ballot CVR is (supposed to be) split 
into as many CCVRs as there are contests on the ballot. 

The CCVRs for the contests are published in C files, 
one for each contest. The CCVR file for contest c should 
contain Nc lines; because this file is published, individu- 
als can check this for themselves. Each line in the CCVR 
file for contest c lists a voter's selection and a shrouded 
version of the identifier of the ballot that the selection 
is supposed to represent. The order of the lines in each 
of the C CCVR files should by shuffled (preferably us- 
ing random permutations) so that whole CVRs cannot be 
reassembled without knowing secret informationFI 

The public can confirm whether the contest outcomes 
according to the CCVR files match the voting system's 
reported outcomes. If they do not match, there should 
be a full hand count of any contests with discrepant out- 
comes. We assume henceforth that the outcomes do 
match, but we do not assume the exact vote totals ac- 
cording to the CCVR files match the reported vote totals. 

The data include one more file that is not published, 
the lookup file. The lookup file contains M lines, one 
for each voting opportunity on each ballot. Each line has 
three entries: a shrouded ballot identifier, the correspond- 
ing unshrouded ballot identifier, and a number ("salt") 
that is used in computing the shrouded identifier from 
the unshrouded identifier using a cryptographic commit- 
ment function, as described be low. (For a rev iew of uses 
for cryptography in voting, see lAdidal 11200611 .) 

The salt on the yth line of the file is denoted Uj. Each 
line corresponds to a (ballot, contest) pair: We can think 
of Uj as being M,f, the salt used to shroud the identity of 
ballot bi in the CCVR file for contest c. The election 
official will use this file to convince observers that every 
selection on every ballot corresponds to exactly one entry 



in a CCVR file, and vice versa. 

3.2 Shrouding 

The method of shrouding ballot identifiers is crucial to 
the approach. SOBA requires election officials to cryp- 
tographically commit to the value of the ballot identifier 
that goes with each CCVR. A cryptographic commitment 
ensures that the ballot identifier is secret but indelible: 
The election official can, in effect, prove to observers that 
a shrouded identifier corresponds to a unique unshrouded 
identifier, but nobody can figure out which unshrouded 
identifier corresponds to a given shrouded identifier with- 
out secret information. 

The next few paragraphs describe a suggested instanti- 
ation of the cryptographic commitment. We assume that 
ballot identifiers all have the same length. If necessary, 
this can be achieved by padding identifiers with leading 
zeros. The commitment function //() must be disclosed 
publicly and fixed for the duration of the election. 

Each commitment represents a claim about a voter's 
selection(s) on a given ballot in a given contest. For 
each set of selections that any voter made in each con- 
test, including undervotes and votes for more than one 
candidate, the election official will create a set of com- 
mitments. Each commitment designates the ballot iden- 
tifier of a ballot that the election official claims contains 
that set of selections in that contest. To commit to the 
ballot identifier b, the election official selects a secret 
"salt" value t|3 and computes the commitment value 
y = H{b,u). At a later stage, the official can open the 
commitment by revealing u and b: Then anyone can ver- 
ify that the value y revealed earlier is indeed equal to 
H{b,u). 

Loosely speaking, a commitment function must have 
two properties, the binding property and the hiding 
property. The binding property makes it infeasible for 
the official to find any pair {b',u') ^ {b,u) for which 
H{b' ,u') = H{b, u). This provides integrity by helping to 
ensure that election officials cannot contrive to have more 
than one CCVR for a given contest claim to come from 
the same ballot^ The binding property is crucial for P- 
resilience; indeed, the proof of P-resilience requires only 
that the commitment have the binding property and that 
{■^t }f=i are known. 

The hiding property makes it infeasible for anyone 
with access only to the shrouded values H{b,u) to learn 
anything about which ballot is involved in each commit- 
ment. This provides privacy by helping to ensure that 



^'For example, each CCVR file could be sorted in order of the 
shrouded ballot identifier. 



^-To protect voter privacy, it must be infeasible to guess the salts: 
Each salt should contain many random or pseudo-random bits. For the 
commitment to be effective, the length of all salt values should be fixed 
and equal. See section|4] 

^^See step 7 of the proof in section l3!4l 
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observers cannot reassemble whole-ballot CVRs from 
the CCVR files without extra information. If observers 
could reassemble whole-ballot CVRs, that would open a 
channel of communication (pattern voting) for coercion 
or vote selling. Ballot identifier b may appear in multiple 
commitments since a separate commitment is generated 
for each candidate selection on each ballot. The hiding 
property ensures that those collections of commitments 
do not together reveal the value of any b. This is crucial 
for the method to be privacy-preserving. 

An HMAC (as described in Federal Information Pro- 
cessing Standard Publication 198) with a secure hash 
function such as SHA-256 (described in Federal Infor- 
mation Processing Standard Publication 180-2) can be 
used to instantiate the commitment function. However, 
since each of the parameters of the commitment function 
is of fixed length it is more efficient to simply use a cryp- 
tographic hash function such as SHA-256 directly. The 
length of the ballot identifiers does not matter, as long as 
all ballot identifiers in the election have the same length. 
We recommend that all salt values have equal length, of 
at least 128 bits. Our results do not depend on the partic- 
ular commitment function chosen, as long as it has both 
the binding and hiding properties^ 

We now describe how to perform a risk-limiting audit 
that simultaneously checks the accuracy of the CCVRs, 
whether each CCVR entry comes from exactly one bal- 
lot, and whether every voting opportunity on every ballot 
is reflected in the correct CCVR file. 

3.3 The audit 

The first three steps check the consistency of the CCVRs 
with the reported results and the uniqueness of the 
shrouded identifiers. 

1 . Verify that, for each contest c, there are A'^ entries 
in the CCVR file for contest c. 

2. Verify that, for each contest c, the CCVR file shows 
the same outcome as the reported outcome. 

3. Verify that the M = A'l H hA^c shrouded ballot 

identifiers in all C CCVR files are unique. 

If step 2 shows a different outcome for one or more con- 
tests, those contests (at least) should be completely hand 
counted. 

Steps 4 and 5 check the logical consistency of the bal- 
lot style file with the reported results. 

4. Verify that, for each contest c, there are Nc entries 
in the ballot style file that list the contest. 



5. Verify that the ballot identifiers in the ballot style 
file are unique. 

If steps 1, 3, 4, or 5 fail, there has been an error or mis- 
representation. The election official needs to correct all 
such problems before the audit can start. 

The remaining steps comprise the statistical portion of 
the risk-limiting audit, which checks whether the CCVRs 
and the mapping from ballots to CCVRs is accurate 
enough to determine the correct winner 

6. Set the audit parameters: 

(a) Choose the risk limit a. 

(b) Choose the maximum number of samples D 
to draw; if there is not strong evidence that the 
outcomes are correct after D draws, the entire 
audit trail will be counted by hand. 

(c) Choose the "error bound inflator" 7 > 1 and 
the error tolerance A G (0,1) for the super- 
simple simultaneous method [StarM l2010bll 
(Y— 1.01 and A = 0.2 are reasonable values). 

(d) Calculate 



log a 



j_ 

2r' 



-Alog(l 



J_) 



(1) 



(e) For each of the C contests, calculate the mar- 
gin of victory in votes from the CCVRs for 
contest cQ 

(f) Calculate the diluted margin jj.: the smallest 
value of nic/N among the C contests!^ 

(g) Calculate the initial sample size «o = [P /m1 • 

(h) Select a seed s for a pseudo-random number 
generator (PRNG)E!| Observers and election 
officials could contribute input values to i or i 
could be generated by an observable, mechan- 
ical source of randomness such as rolls of a 
10-sided die. The seed should be selected only 
once. 

7. Draw the initial sample by finding no pseudo- 
random numbers between 1 and and audit the cor- 
responding ballots: 



^ iMenezes Il99ell offers a thorough treatment of hash func- 
tions and their use for commitments in applications such as digital sig- 
natures. 



^^This would be re placed by a differe n t calculation for IRV or RCV 
contests. See. e.g.. iMagrino et all |201 lll i lCmyl |201 ill . 

^'The diluted margin controls the sample size. If contest c has the 
smallest value of mdN and is rather smaller than N, it can be more 
efficient to audit contest c separately rather than auditing all C contests 
simultaneously. 

^'The code for the PRNG algorithm should be published so that it 
can be checked and so that, given the seed .v, observers can reproduce 
the sequence of pseudo-random numbers. The PRNG should produce 
numbers that are statistically indistinguishable from independent ran- 
dom numbers uniformly distributed between and 1 (i.e., have large 
p-values) for sample sizes up to millions for a reasonable battery of 
tests of randomness, such as the Diehard tests. 
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(a) Use the PRNG and the seed s to generate no 
pseudo-random numbers, ri,r2, . . . ,r„Q. 

(b) Let £j = \Nrj~\, ; = 1, . . . ,no- This Ust might 
contain repeated values. If so, the tests be- 
low only need to be performed once for each 
value, but the results count as many times as 
the value occurs in the listF^ 

(c) Find rows ii,... , £no in the ballot style file. 

(d) Retrieve the ballots bf. in the audit trail iden- 
tified by those rows in the ballot style file. If 
there is no ballot with identifier b(j, pretend in 
step 7(g) below that the ballot showed a vote 
for the runner-up in every contest listed in that 
row of the ballot style file. 

(e) Determine whether each ballot shows the 
same contests as its corresponding entry in the 
ballot style file. If there are any contests on the 
ballot that are not in the ballot style file entry, 
pretend in step 7(g) below that the CCVR for 
that (ballot, contest) pair showed a vote for the 
apparent winner of the contest. If there are any 
contests in the ballot style file entry that are 
not on the ballot, pretend in step 7(g) below 
that the ballot showed a vote for the apparent 
runner-up for that contest. 

(f) For each ballot bfj in the sample, the election 
official reveals the value of u^jc for each con- 
test c on the ballot. 

(g) For each ballot in the sample, for each contest 
on that ballot, observers calculate H{bi,.,ui,.c) 
and find the entry in the CCVR file for con- 
test c that has that shrouded identifier If the 
shrouded identifier is not in the CCVR file, 
pretend that the CCVR file showed that the 
voter had selected the apparent winner of con- 
test c. Compare the voter's selection(s) ac- 
cording to the CCVR file to the voter's selec- 
tion(s) according to a human reading of ballot 
b(.. Find e(j, the largest number of votes by 
which any CCVR for ballot b^. overstated the 
margin between any (winner, loser) pair in any 
contest on ballot b( . . This number will be be- 
tween —2 and +2. 

8. If no ballot in the sample has e(. ~ 2 and no more 
than A/ino have Cf,. = 1, the audit stops. (In this 
calculation, the value of e^. should be counted as 
many times as ij occurs in the sample.) 

9. Otherwise, calculate the Kaplan- Markov P- valu e, 
Pkm according to equation (9) in IStark Il2009dll3. 



2010bll FI If Pkm is less than a, the audit stops. If 
Pkm is greater than a, the sample is expanded: An- 
other random number rj is generated and steps 7(c)- 
(g) are repeated. The value of Pkm is updated to 
include the overstatement errors found in the new 
draw0 This continues until either Pkm <(Xov there 
have been D draws. In the latter case, all remaining 
ballots are counted by hand, revealing the true out- 
come. 



The next section establishes that this procedure in fact 
gives a risk-limiting audit. 

3.4 Proof of the risk-limiting property 

If the ballot style file is correct and entries in the CCVR 
files are mapped properly to voting opportunities on ac- 
tual ballots, the only potential source of error is that 
CCVR entries do not accurately reflect the voters' selec- 
tions according to a human reading of the ballot. If that 
is the case, this is an "ord inary" risk-limiting audit, and 
the proof inl Stark I 201 0b*l that the super-simple simulta- 
neous method is risk-limiting applies directly. 

Suppose therefore that the ballot style file or the map- 
ping between ballots and CCVRs is faulty. Recall that 
the super-simple simultaneous method assumes that no 
ballot can overstate any margin by more than ly votes, 
where 7 > 1 . There are seven cases to consider 



1. The ballot style file has more than one entry that 
corresponds to the same actual ballot, or more than 
one actual ballot corresponds to the same entry in 
the ballot style file. These faults are precluded by 
the uniqueness of the ballot identifiers and of the 
recipes for locating the actual ballot with each iden- 
tifier. 

2. More than one ballot identifier corresponds to the 
same shrouded entry (for different values of u). This 
is precluded by the binding property of H. 



^'We consider only plurality voting here: IRV is more complicated. 
For each contest c, let Wc be the indices of the apparent winners of the 
contest and let J^c be the indices of the apparent losers of the contest. 
IfwE'^c and x e let Vu-x be the margin in votes between candidate 
w and candidate x according to the CCVR file for contest c. For each 
candidate k on ballot (, let V(i( denote the number of votes for candidate 
k on ballot £ according to the CCVR file and let denote the number 
of votes on ballot f. for candidate k according to a human reading of 
ballot e. Let 

et = max max (vc,^ - (if,,, - + afa)/V,v.v (2) 



Then 



_A 1-1/t/ 

l^KM = 11 —■ 



(3) 



^^The auditing method relies on sampling with replacement to limit 
the risk. 



^"Overstatements are calculated as step 7 above, including, in partic- 
ular, steps 7(e) and 7(g), which say how to treat failures to find ballots 
or contests. 
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3. The ballot style file contains identifiers that do not 
correspond to actual ballots, or claims that a ballot 
contains a contest that it does not actually contain. 
The biggest effect this could have on an apparent 
contest outcome is if the ballot that entry is sup- 
posed to match showed a vote for the runner-up in 
every missing contest, which is no greater than a 
two-vote change to any margin. Because the au- 
dit samples entries of the ballot style file with equal 
probability, this kind of error in an entry is just as 
likely to be revealed as any other If such a ballot 
style file entry is selected for audit, steps 7(d) and 
7(e) treat it this worst-case way. 

4. The ballot style file claims that a ballot does not 
contain a contest that it does contain. The biggest 
effect this could have on an apparent contest out- 
come is if the CCVR for that contest showed a vote 
for the apparent winner, which cannot change the 
margin by more than two votes, so the error-bound 
assumptions are satisfied. Because the audit sam- 
ples entries of the ballot style file with equal proba- 
bility, this kind of error in an entry is just as likely 
to be revealed as any other. If such a ballot style 
file entry is selected for audit, step 7(e) treats it this 
worst-case way. 

5. There are ballots whose identifiers do not appear in 
the ballot style file. Since there are the same number 
of ballots as entries in the ballot style file and the 
ballot identifiers in the ballot style file are unique, 
there must be ballot identifiers in the ballot style file 
that do not match any ballot. Hence, case (3) holds. 

6. There are CCVRs for which the shrouded ballot 
identifier is not the identifier of any ballot. If the 
shrouded identifier matches an identifier in the bal- 
lot style file, we are in case (3). Suppose therefore 
that the shrouded identifier does not match any in 
the ballot style file. Suppose this happens for con- 
test c. The preliminary checks show that the bal- 
lot style file has exactly A^^- entries for contest c and 
that there are exactly Nc entries in the CCVR file 
for contest c. Therefore, if there is such a CCVR, 
one of the ballot style file entries that lists contest 
c has an identifier that does not occur in shrouded 
form in the CCVR file for that contest. The largest 
effect this could have on contest c is if the "substi- 
tuted" CCVR entry reported a vote for the apparent 
winner; this cannot overstate the margin by more 
than two votes, so the audit's error-bound assump- 
tion still holds. Because the audit samples entries of 
the ballot style file with equal probability, this kind 
of error in a ballot style file entry is just as likely 
to be revealed as any other. If such a ballot style 



file entry is selected for audit, step 7(e) treats it this 
worst-case way. 

7. The same ballot identifier appears in shrouded form 
more than once in a single CCVR file. As in the 
previous case, we know there are A^^ entries in the 
CCVR file for contest c and A'^ entries in the bal- 
lot style file that include contest c; moreover, the 
identifiers in the ballot style file are unique. Hence, 
there must be at least one entry in the ballot style 
file that lists contest c for which the ballot identifier 
does not appear in shrouded form in the CCVR file. 
We are therefore in case (6). 

4 Discussion 

Others have proposed election verification methods 
that involve a cryptographic commitment by elec- 
tions officials to a mapping between ballots and 
CVRs [E.K. Rescorla, personal communication, 2011; 
R.L. Rivest, personal communication, 20 09; D. Wallac h, 
personal communication, 2010; see also lAdidal ll2006ll l. 
However, we believe SOBA is the first method that re- 
quires only one commitment and that uses a risk-limiting 
audit to check whether the mapping is accurate enough 
to determine the correct winner 

We have said little about the requirement for a com- 
pliance audit. In part, this is a definitional issue: Even if 
the audit trail is known to have been compromised, it is 
our understanding that in many states, a full hand count 
of the audit trail would still be the "correct" outcome, as 
a matter of law. Hence, an audit to assess whether the 
audit trail was protected and preserved adequately for it 
to reflect the outcome according to how the voters cast 
their ballots is legally superfluous. We consider this a 
shortcoming of current audit and recount laws. More- 
over, we doubt that any system can be P-resilient unless 
the election and the data it generates satisfies particular 
conditions. For instance, risk-limiting audits generally 
assume that the number of ballots cast in all in each con- 
test is known. Such conditions should be checked. 

We would advocate carrying out a compliance au- 
dit to assess whether the procedures as followed in the 
election give reasonable assurance that the audit trail is 
trustworthy — sufficiently accurate to reflect the outcome 
according to how voters cast their ballots — and to assess 
whether any other preconditions of the risk-limiting au- 
dit hold. The compliance audit should evaluate whether 
there is strong evidence that the chain of custody of the 
ballots is intact, or whether it is plausible that ballots 
were lost, "found," altered, or substituted. The compli- 
ance audit should confirm the values of {Nc} by bal- 
lot accounting: confirming that the number of ballots 
printed equals the number returned voted, unvoted, and 
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spoiled, for each ballot type. 

If the election passes the compliance audit, a risk- 
limiting audit can then assess the accuracy of the reported 
result and would have a large chance of correcting the ap- 
parent outcome if it is wrong (by examining the full audit 
trail). But if the election fails the compliance audit — that 
is, if we lack strong evidence that the audit trail is reli- 
able and that the preconditions for the risk-limiting audit 
are met — a P-resilient election framework should not de- 
clare any outcome at all. 

For the method to be f -resilient, H must be binding 
and we must know {Nc}. Because the election official 
discloses H and the (fixed) length of the ballot identi- 
fiers, we can determine whether H is binding. For the 
method to be privacy-preserving, H must have the hiding 
property, which will depend on how the salts are chosen 
and how the CCVR files are organized. If the salts can 
be discovered, inferred, or guessed, or if observers have 
another way to reassemble whole-ballot CVRs from the 
CCVRs (for instance, if the CCVRs are in the same bal- 
lot order across contests), voter privacy can be compro- 
mised. 

5 Conclusions 

SOBA makes possible a personally verifiable privacy- 
preserving P-resilient canvass framework. It allows indi- 
viduals to obtain strong firsthancj^l] evidence that appar- 
ent election outcomes either are correct in the first place, 
or are corrected by a risk-limiting audit before becoming 
final, without unnecessary compromises to privacy. Af- 
ter the procedure is complete, either all the outcomes are 
correct or an event with probability less than \ — P has 
occurred. The published data structures allow the pub- 
lic to check the consistency of the apparent outcomes but 
do not allow whole-ballot cast vote records to be recon- 
structed, thereby preserving privacy. When all the appar- 
ent contest outcomes are correct, gathering the evidence 
that the outcomes are right typically will require expos- 
ing only a small fraction of ballots to observers, protect- 
ing privacy. But the data structures and auditing protocol 
ensure that if the apparent outcome of one or more of the 
contests is wrong, there is a large chance of a full hand 
count of the audit trail to set the record straight. 
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